Knowledge extraction from Copernicus satellite data

We describe two alternative approaches of how to extract knowledge from high- and medium-resolution Synthetic Aperture Radar (SAR) images of the European Sentinel-1 satellites. To this end, we selected two basic types of images, namely images depicting arctic shipping routes with icebergs, and - in contrast - coastal areas with various types of land use and human-made facilities. In both cases, the extracted knowledge is delivered as (semantic) categories (i.e., local content labels) of adjacent image patches from big SAR images. Then, machine learning strategies helped us design and validate two automated knowledge extraction systems that can be extended for the understanding of multispectral satellite images

Deep Learning Training and Benchmarks for Earth Observation Images: Data Sets, Features, and Procedures

Deep learning methods are often used for image classification or local object segmentation. The corresponding test and validation data sets are an integral part of the learning process and also of the algorithm performance evaluation. High and particularly very high-resolution Earth observation (EO) applications based on satellite images primarily aim at the semantic labeling of land cover structures or objects as well as of temporal evolution classes. However, one of the main EO objectives is physical parameter retrievals such as temperatures, precipitation, and crop yield predictions. Therefore, we need reliably labeled data sets and tools to train the developed algorithms and to assess the performance of our deep learning paradigms. Generally, imaging sensors generate a visually understandable representation of the observed scene. However, this does not hold for many EO images, where the recorded images only depict a spectral subset of the scattered light field, thus generating an indirect signature of the imaged object. This spots the load of EO image understanding, as a new and particular challenge of Machine Learning (ML) and Artificial Intelligence (AI). This chapter reviews and analyses the new approaches of EO imaging leveraging the recent advances in physical process-based ML and AI methods and signal processing

Dialectical GAN for SAR Image Translation: From Sentinel-1 to TerraSAR-X

Contrary to optical images, Synthetic Aperture Radar (SAR) images are in different electromagnetic spectrum where the human visual system is not accustomed to. Thus, with more and more SAR applications, the demand for enhanced high-quality SAR images has increased considerably. However, high-quality SAR images entail high costs due to the limitations of current SAR devices and their image processing resources. To improve the quality of SAR images and to reduce the costs of their generation, we propose a Dialectical Generative Adversarial Network (Dialectical GAN) to generate high-quality SAR images. This method is based on the analysis of hierarchical SAR information and the "dialectical" structure of GAN frameworks. As a demonstration, a typical example will be shown where a low-resolution SAR image (e.g., a Sentinel-1 image) with large ground coverage is translated into a high-resolution SAR image (e.g., a TerraSAR-X image). Three traditional algorithms are compared, and a new algorithm is proposed based on a network framework by combining conditional WGAN-GP (Wasserstein Generative Adversarial Network - Gradient Penalty) loss functions and Spatial Gram matrices under the rule of dialectics. Experimental results show that the SAR image translation works very well when we compare the results of our proposed method with the selected traditional methods.Comment: 22 pages, 15 figure

An Active Learning Tool for the Generation of Earth Observation Image Benchmarks

This paper describes an active learning tool for the genera-tion of Earth Observation (EO) benchmark datasets. This toolis able to generate training datasets, based on its active learn-ing strategy with a classification accuracy of around 90%.Afterwards, a data cleaning tool is needed, in order to cor-rect noisy data and provide a clean dataset to be stored in thebenchmark database, and for subsequent benchmark verifica-tion. The data cleaning procedure is supported by unsuper-vised learning, using clustering algorithms to group similarpatterns, and dimension reduction algorithms to embed themin lower dimension with annotated labels. Moreover, interac-tive visualizations are implemented in most modules to helpbetter manipulate datasets and get better understandings

Semantic Labelling of Globally Distributed Urban and Non-Urban Satellite Images Using High Resolution SAR Data

While the analysis and understanding of multispectral (i.e., optical) remote sensing images has made considerable progress during the last decades, the automated analysis of SAR (Synthetic Aperture Radar) satellite images still needs some innovative techniques to support non-expert users in the handling and interpretation of these big and complex data. In this paper, we present a survey of existing multispectral and SAR land cover image datasets. To this end, we demonstrate how an advanced SAR image analysis system can be designed, implemented, and verified that is capable of generating semantically annotated classification results (e.g., maps) as well as local and regional statistical analytics such as graphical charts. The initial classification is made based on Gabor features and followed by class assignments (labelling). This is followed by the inclusion. This can be accomplished by the inclusion of expert knowledge via active learning with selected examples, and the extraction of additional knowledge from public databases to refine the classification results. Then, based on the generated semantics, we can create new topic models, find typical country-specific phenomena and distributions, visualize them interactively, and present significant examples including confusion matrices. This semi-automated and flexible methodology allows several annotation strategies, the inclusion of dedicated analytics procedures, and can generate broad as well as detailed semantic (multi-)labels for all continents, and statistics or models for selected countries and cities. Here, we employ knowledge graphs and exploit ontologies. These components could already be validated successfully. The proposed methodology can also be adapted to other instruments

Machine Learning Techniques for Knowledge Extraction from Satellite Images: Application to Specific Area Types

When we want to extract knowledge form satellite images, several well-known image classification and analysis techniques can be concatenated or combined to gain a more detailed target understanding. In our case, we concentrated on specific extended target areas such as polar ice-covered surfaces, forests shrouded by fire plumes, flooded areas, and shorelines. These image types can be described by characteristic features and statistical relationships. Here, we demonstrate that both multispectral (optical) as well as SAR (Synthetic Aperture Radar) images can be used for knowledge extraction. The free availability of image data provided by the European Sentinel-1 and Sentinel-2 satellites allowed us to conduct a series of experiments that verified our classification approaches. This could already be verified in our recent work by quantitative quality tests

Earth Observation Semantics and Data Analytics for Coastal Environmental Areas

Current satellite images provide us with detailed information about the state of our planet, as well as about our technical infrastructure and human activities. A range of already existing commercial and scientific applications try to analyze the physical content and meaning of satellite images by exploiting the data of individual, multiple or temporal sequences of images. However, what we still need today are advanced tools to automatically analyze satellite images in order to extract and understand their full content and meaning. To remedy this exploration problem, we outline a highly automated and application-adapted data-mining and content interpretation system consisting of five main components, namely Data Sources (selection and storage of relevant images), Data Model Generation (patch cutting and generation of feature vectors), Database Management System (systematic data storage), Knowledge Discovery in Databases (clustering and content labeling), and Statistical Analytics (generation of classification maps). As test sites, we selected UNESCO-protected areas in Europe that include coastal areas for monitoring and an area known in the Mediterranean Sea that contains fish cages. The analyzed areas are: the Curonian Lagoon in Lithuania and Russia, the Danube Delta in Romania, the Hardangervidda in Norway, and the Wadden Sea in the Netherlands. For these areas, we are providing the results of our image content classification system consisting of image classification maps and additional statistical analytics based on three different use cases. The first use case is the detection of wind turbines vs. boats in the Wadden Sea. The second use case is the identification of fish cages/aquaculture along the Mediterranean coast. Finally, the third use case describes the differences between beaches, dams, dunes, and tidal flats in the Danube Delta, the Wadden Sea, etc. The average classification accuracy that we obtained is ranging from 80% to 95% depending on the type of available images